The present invention relates to processing of video signals. In particular, this invention relates to a method of de-interlacing interlaced video formats using a mixed mode spatial and temporal approximation technique.
Video signals are currently represented as sequences of a) fields in case of interlace scan refresh or b) frames according to non-interlace or progressive scan refresh. In the interlaced scan format, a single image (frame) is represented using a pair of fields. One field of the pair features pixels located in alternate rows (odd numbered horizontal lines, for example) of the field matrix. The second field of the pair features pixels located in the same field matrix only in the corresponding horizontal lines (even numbered horizontal lines, for example) which were missing pixels in the first field, such that portions of the image not represented in the first field are represented in the second field. In the interlaced scan format, each field of image data is scanned twice, once for the odd numbered horizontal lines of the field, and another time for the even numbered horizontal lines of the field, in order to have all of the horizontal lines of the odd field followed by all of the horizontal lines of the even field. The pair of fields of odd and even horizontal lines in interlaced video constitute the frame (one full resolution picture or image). In contrast, in the de-interlaced or progressive scan format, an image is represented in its entirety using only a single field which includes pixels in all horizontal lines of the field matrix. Here, each frame (field) of image data is scanned once from the top horizontal line to the bottom horizontal line without requiring interlacing action between two fields.
In the interlace scan format, the first and second fields of a pair are scanned consecutively on a video display monitor at a rate of 60 fields per second, in order to reconstruct single image frames on the display at the industry interlaced scan standard of 30 frames per second. In more recently developed video representation technique using de-interlacing (progressive scan format) format, frames are progressively scanned on a display at the standard progressive display rate of 60 frames per second.
Application of current interlace scan format to television, includes the NTSC (National Television System Committee) and the PAL (Phase Alternation by Line) systems. In the NTSC format, there are 262.5 horizontal scanning lines per field (including one odd numbered field, and one even numbered field), translating to 525 scanning lines per frame, with an established scan rate of (60 fields) 30 frames per second. In the PAL format, there are 312.5 horizontal scanning lines per field (including one odd numbered field, and one even numbered field), translating to 625 scanning lines per frame, with an established scan rate of (50 fields) 25 frames per second.
New display systems such as CRT (PC monitors), flat liquid crystal device (LCD) panels, plasma display panels (PDP), and video equipment, including cameras, broadcast station transmitters and high definition television (HDTV) desktop or workstation display monitors are using de-interlaced (progressive) high resolution scan format systems such as VGA(480 lines.times.640 columns per frame), SVGA(600 lines.times.800 columns per frame), XGA(768 lines.times.1024 columns per frame), and UXGA(1200 lines.times.1600 columns per frame) to scan and display image data. An example showing the need for de-interlacing interlaced video data, is a typical LCD display having 480 horizontal scanning lines with 640 dots per scanning line (VGA system). Since LCD display systems are designed to be scanned in the de-interlaced format, when the need is to display NTSC (525 lines per frame) and PAL (625 lines per frame) image signals on an LCD display, interlaced image signals need to be converted into de-interlaced image signals for proper display.
It is known that higher quality image reproductions are obtained by using de-interlaced scanned format rather than interlaced scan format, because interlaced displays are more likely to exhibit visual artifacts (such as line crawl on diagonal edges of an image, and interline flicker on horizontal edges of an image) than de-interlaced scan displays. As a result, there has been substantial effort towards developing methods of converting or de-interlacing interlaced video image data suitable for display on de-interlaced or progressive scan format devices.
Several conversion or de-interlacing methods, devices, and systems for video image processing have been developed, most of which feature one or a more of a variety of spatial, temporal, or spatio-temporal interpolation processing for estimating the values of missing pixels in an interlaced frame. The relative suitably of these techniques depends on the resulting image quality. Moreover, different interpolation techniques and systems work better under different conditions.
U.S. Pat. No. 5,661,525 issued to Kovacevic et al., features a method for de-interlacing an interlaced video frame sequence using interpolation of spatial and temporal pixels for estimating values of missing pixels. The interpolations are weighted according to the errors each one introduces for generating the approximations of missing pixel values for a de-interlaced frame. In these interpolations, three fields of pixels are used, i.e., the current spatial field and the two neighboring (immediately preceding and following) fields of pixels are used for estimating values of missing pixels in the current spatial field. In U.S. Pat. No. 5,793,435 issued to Ward et al., a de-interlacing system for converting an interlaced video to a progressive video features a variable coefficient, non-separable spatio-temporal interpolation filter. Reference and offset video signals are weighted together with filter coefficients in the spatio-temporal interpolation filter, to produce an interpolated video signal. The interpolated video signal is interleaved with the reference video signal, suitably delayed to compensate for filter processing time, to produce the de-interlaced video signal. U.S. Pat. No. 5,621,470 issued to Sid-Ahmed makes use of interpolation in an inter-pixel and inter-frame arrangement and incorporates a 3D (low pass) filter to support such actions. The 3D interpolator produces twice the number of pixels along each horizontal line, twice the number of lines in each frame and double the number of frames per second. Another interpolation filter apparatus applied to de-interlacing is presented in U.S. Pat. No. 5,559,905 issued to Greggain et al., in which interpolation of a stream of input pixels involves a filter providing a means for aligning the stream of input pixels and the first derived stream of sampled (output) pixels at the boundaries of the image, at a predetermined interpolation rate. U.S. Pat. No. 5,650,824 issued to Si Jun Huang, includes the use of a filter which performs linear interpolation on interlaced image data, involving two neighboring field samples for each odd field and even field input.
An example of a device for implementing an interpolation method of de-interlacing video signals is given in U.S. Pat. No. 5,717,466 issued to Shao Wei Pan et. al., featuring an enhanced video circuit for performing (linear and non-linear) non-uniform interpolation of video scan lines. A real-time video system which incorporates the circuit device featured in U.S. Pat. No. 5,717,466, is shown in U.S. Pat. No. 5,742,350, issued to the same.
Current methods of de-interlaced video signals are notably limited with respect to de-interlacing video images featuring textual data. Standard approximation methods currently used for de-interlacing interlaced video signals are typically based on interpolation techniques, for evaluating missing pixels in interlaced fields of video signals. These interpolation techniques require the use of no less than three fields of pixels with known values for estimating values of missing pixels.
A more accurate and comprehensive approximation method for de-interlacing interlaced video signals involves usage of logical operations, for making decisions leading to assignment of highly accurate values of missing pixels, included in a technique which involves extrapolation, and not only interpolation, of missing pixels in interlaced fields of video signals. Moreover, a de-interlacing method which requires less than three fields of pixels with known values for approximating values of missing pixels would translate to a significant savings of resources required for de-interlacing. A method requiring input information from two, instead of three, fields of pixels with known values would require measurably less data processing resources including hardware, software, memory, and calculation time. There is thus a need for, and it would be useful to have, an accurate and comprehensive method of de-interlacing interlaced video signals currently used in standard video and television devices, which is generally applicable to both numerical and textual image data, and which requires fewer resources. Moreover, there is a need for such an improved de-interlacing method applicable to either real time or off-line mode of operation of video and television signal de-interlacing.